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.Traditional  moderated  regression  analysis  requires  a 
increment  in  R2  due  to  the  interaction  term  in  order 
a moderator  effect.  A Monte  Carlo  analysis  dcuionstr 
procedure  will  not  discover  moderators  even  when  the 
effect.  A suggestion  is  made  to  reverse  the  usual  h 
moderated  regression  analysis.  The  choice  between  a 
model  should  be  made  on  the  basis  of  theoretical  jus 
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Whore  Have  All  the  Moderators  Gone? : 


The  Perils  of  Type  II  Error  ; 

Moderated  regression  analysis  has  been  used  in  psychological  research 
since  its  presentation  by  Saunders  (1956).  This  technique  assesses  the 
influence  of  a third  variable,  called  a moderator,  on  the  relationship 
between  two  other  variables.  Knowledge  of  the  level  of  the  moderator 

i 

variable  provides  information  about  the  strength  of  the  relationship  between 
the  other  two  variables.  In  his  explication  of  the  procedure,  Saunders 
pointed  out  that  a moderator  effect  will  manifest  itself  as  a relationship 
between  the  dependent  variable  and  the  cross  product  of  the  independent  and 
moderator  variables.  Conceptually,  this  is  equivalent  to  saying  that  the 
relationship  between  the  dependent  and  independent  variables  is  influenced 
by  the  moderator  variable. 

The  relationship  thus  described  allows  the  postulation  of  individual 
differences  in  the  relationships  between  variables,  i.e.,  one  could  hypothesize 
that  the  relationship  between  two  variables  is  stronger  for  individuals  who 
are  high  on  a third  characteristic  than  for  individuals  who  are  low  on  the 
third  characteristic.  For  example,  in  the  original  article  Saunders  (1956) 
tested  the  hypothesis  that  the  rel ationship  between  the  Engineer  Scale  of  the 
Strong  Interest  Blank  (independent  variable)  and  the  grade  point  average 
(dependent  variable)  of  engineering  freshman  is  higher  for  non-compulsive 
(moderator  variable)  students  than  for  compulsive  students.  This  pattern 
of  interrelationships  among  variables  lias  widespread  usefulness  in  model 
building.  As  stated  by  Saunders  (1956),  "The  class  of  situations  in  which 
the  'moderated  multiple  regression'  might  he  profitably  studied  can  be  made. 
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quite  large,  and  can  be  seen  to  include  a number  of  situations  of  potential 
practical  significance"  (p.  210). 

The  concept  of  moderators  has  been  utilized  in  a number  of  subsequent 
theories  and  moderated  regression  has  been  empirically  investigated  in  a 
variety  of  settings.  The  steps  of  the  traditional  procedure  for  empirically 
determining  the  existence  of  a moderator  effect  are  presented  by  Zedeck  (1971). 
"The  correlations,  slopes,  and  standard  error  of  estimates  should  be  examined 
for  the  following  three  regression  equations: 

y = a + bx,  [i 

y ~ a + bjX  4-  b^z  [2 

where  z is  the  potential  moderator  but  is  treated  as  an  independent  predictor, 
and 

y <=  a + bjX  + b^z  + b^xz  [3 

(moderated  regression  equation).  If  equations  2 and  3 are  significantly  different 

from  equation  1,  but  not  from  each  other,  then  the  variable  is  an  independent 

predictor  and  not  a moderator  variable"  (Zedeck,  1971,  p.  304).  These  suggested 

steps  have  been  followed  whenever  a moderated  regression  analysis  has  been 

conducted.  Cohen  (1968)  and  Cohen  and  Cohen  (1973)  show  that  moderated 

regression  is  one  conceptualization  of  an  interaction  effect  in  the  general 

linear  model.  They  describe  a hierarchical  analysis  for  assessing  the  existence 

of  an  interaction  that  is  equivalent  to  the  procedure  outlined  above,  i.e.,  one 

2 

•first  determines  the  squared  multiple  correlation  coefficient;  (R  ) for  model 

2 

2 and  then  determines  the  increment  in  R caused  by  adding  the  interaction  term 
in  model  3.  The  increment*  is  then  tested  for  statistical  significance,  using 
model  3 error  mean  square. 
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In  this  hierarchical  treatment,  the  moderator  effect  must  account  for 
dependent  variable  variance  beyond  that  accounted  for  by  the  main  effects  alone. 
The  main  effects  are,  thus,  give.,  a preferential  claim  to  any  covariance  that  is 
shared  by  main  effects  and  the  moderator  effect.  Such  preference  may  be 
unreasonable  under  the  conditions  that  generally  hold  when  a moderated  regression 
analysis  is  conducted.  The  moderated  regression  analysis  is  usually  done  only 
if  there  is  some  theoretical  basis  to  suspect  the  presence  of  a moderator 
effect.  'If  such  is  the  case,  the  researcher  should  attempt  to  use  an  analysis 
with  reasonable  statistical  power,  i.e.,  the  probability  of  discovering  an 
existing  hypothesized  effect.  This  article  will  demonstrate  that  traditional 
moderated  regression  analysis  procedure,  on  the  contrary,  enhances  the 
possibility  of  type  II  error,  i.e.,  the  failure  to  recognize  the  presence  of  an 
existing  effect. 


This  subtle  but  critical  flaw  in  the  traditional  analysis  procedure  can 
explain  why  the  results  of  moderated  regression  analyses  led  Zedeck  (1971)  to 


state  "In  general,  moderated  regression  has  not  been  successful  in  improving 
predictions"  (p.  302).  Competitive’  demands  fen  journal  space  and  the  tendencies 
of  researchers  to  report  only  significant  finds  allows  speculation  that  many 
instances  of  failure  to  find  moderated  regression  have  gone  unreported.  These 
failures  to  discover  moderator  effects  and  the  disappointing  characteristics 
of  those  effects  that  hove  been  found  result  from  the  features  of  the  traditional 
analysis  procedure  that  are  demonstrated  in  the  data  below. 

Among  the  reported  studies  that  have  used  moderated  regression  analysis, 
the  results  have  been  quite  similar  but  the  conclusions  drawn  from  them  have 
changed  from  one  study  to  another.  Investigators  have  based  their  conclusions 
on  two  statistical  characteristics  of  the  moderated  relationship — the  statistical 
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significance  of  the  increment  due  to  the  interaction  term  and  the  size  of  the 
increment . 

Requirement  o f St: at  i stical  Signif  lean ce 

As  examples  of  the  moderated  regression  analysis  procedure,  Saunders  (1956) 

presented  three  sets  of  data.  In  his  first  example,  only  one  moderator  effect 

2 

was  investigated.  The  increment  in  R was  .013,  not  significant.  A second 
set  of  analyses  tested  the  moderator  effects  of  ten  potential  moderators.  In 
this  case,  the  increments  ranged  from  zero  to  .025  with  three  of  the  ten 
reaching  the  .05  level  of  significance.  His  third  set  of  demonstration  data 
tested  seven  potential  moderator  effects.  The  increments  ranged  from  zero  to 
.017  with  one  of  the  effects  significant  at  the  .01  level.  He  concluded  that  a 
variable  was  operating  as  a moderator  variable  whenever  the  increment  reached 
signif icance. 

Jacobs  and  Solomon  (in  press)  also  concluded  that  a moderator  effect  had 
2 

been  shown  when  the  K*  increment  was  statistically  significant.  In  their  case 

2 

11  or  12  moderator  effects  were  significant  beyond  the  .05  level.  The  increments 
2 

in  R ranged  from  .009  to  .104.  In  both  the  Saunders  (1956)  and  Jacobs  and 
Solomon  (in  press)  studies  the  criterion  for  concluding  that  there  is  a moderator 
is  the  one  stated  by  Zcdeck,  Cranny,  Vale,  and  Smith  ( 1 97 1) > "Presence  of  a 
moderator  is  thus  indicated  by  the  form  of  the  equation,  specifically  by  the 
significance  of  cross-product  terms"  (p.  239). 

Requirement  of  a Sizable  Increment 

Stone  (1976)  and  Stone,  Mowday,  and  Porter  (in  press)  drew  a different 

2 

conclusion  from  similar  results.  Stone  (1976)  showed  an  increment  in  R of 
.016  significant  at  the  .001  level.  While  acknowledging  the  statistical 
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significance  of  the  moderator  effect,  lie  went  on  to  conclude  from  the  size  of 

the  increment  that  "the  gain  in  explained .. .variance  is  not  marked  when  the... 

interaction  term  is  introduced  into  the  multiple  prediction  equation.  The 

practical  significance. . .appears,  therefore,  to  be  negligible"  (p.  163).  In  a 

2 

discussion  of  a finding  of  an  increment  in  R of  .019,  significant  at  the 

.01  level,  Stone,  Mowday,  and  Porter  (in  press)  state  that  "the  difference 

between  the  two  multiple  correlations  while  statistically  significant, .. .is 

of  negligible  size"  (p.  8).  In  both  of  these  cases  the  authors  have  taken  the 

2 

size  of  the  increment  in  R as  an  index  of  the  strength  of  the  moderator  effect . 

If  a large  increment  is  a legitimate  requirement  for  drawing  the  conclusion  that 

a moderator  should  be  attended  to,  then  one  can  wonder  if  any  moderated  regression 

analysis  has  ever  produced  a moderator  effect  that  was  not  "negligible." 

2 

Further,  if  the  size  of  the  increment  in  R is  taken  as  an  indication  of  the 

strength  of  the  moderator  effect,  the  uniformly  modest  effects  t hat  have  been 

reported  in  the  literature  would  argue  that  moderator  effects  arc  not  important 

tools  in  scientific  endeavors,  i.e.,  conceptual  models  that  include  moderator 

effects  could  just  as  well  do  without  them.  This  article  will  argue  that  the 

2 

size  of  the  increment  in  R cannot  be  interpreted  in  that  fashion. 

The  research  reported  in  this  article  was  conducted  to  test  the  efficiency 
of  the  traditional  moderated  regression  analysis  procedures  for  discovering 
moderators.  The  study  considers  whether  the  existence  or  the  strength  of 
moderator  effects  is  accurately  reflected  in  the  results  from  the',  traditional 
moderated  regression  analysis. 


Method 


Da  lh  Generation 


In  order  to  simulate  data  on  which  to  perform  the  moderated  regression 
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analysis  using  a Monte  Carlo  approach,  random  normal  deviates  were  generated 

according  to  the  procedures  of  Box  and  Muller  (1958).  In  all  cases  three 

variables,  x,  z,  and  e were  constructed  for  each  synthetic  "individual." 

Variable  x was  treated  in  subsequent  analyses  as  the  independent  variable,  z 

was  treated  as  the  moderator  variable,  and  e was  used  as  error.  The  dependent 

variable,  y,  was  constructed  as  described  below  for  various  cases.  Except 

where  it  is  noted  in  the  definitions  of  the  various  cases  given  below,  x and 

3 

z had  means  - 10  and  standard  deviations  - 1 ; e had  a mean  - 0 and  a standard 

deviation  - 1. 

Case  One.  In  the  first  case  the  dependent  variable  was  constructed  so  as 
to  represent  a situation  with  a moderator  effect  and  some  error:  The  generating 

model  was  y = xz  +3.  One  hundred  samples,  each  with  100  subjects,  were 
constructed.  This  case  ‘served  as  the  basic  moderator  effect  case,  and  other 
cases  were  generated  for  comparison  purposes.  In  the  additional  cases,  changes 
were  made  in  the  formula  for  generating  the  dependent  variable. 

Case  Two . The  second  case  represents  a situation  with  greater  unreliability 
in  the  measurement . This  effect  was  accomplished  by  increasing  the  variance  of 
the  error  term.  The  same  model  (y  - xz  + e)  was  used  to  generate  the  dependent 
variable.  For  this  case  the  standard  deviation  of.  the  error  variable  ~ 5.  TWenty 
samples,  each  with  1 00  subjects,  were  generated. 

Case  Three.  Hie  third  case  also  used  the  basic  model  (y  = xz  + e)  to 
create  a dependent  variable  from  a moderator  effect  . Unlike  cases  one  and  two, 
the  independent  variable  and  the  moderator  variable  were  constructed  to  be 
correlated.  This  relationship  varied  because  of  sample  differences  between  .Ah 
and  1.00  with  a median  value  of  .55.  'iVenty  samples,  each  with  100  subjects, 
were  generated.  , 
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Casa  Four.  In  the  fourth  case  the  dependent  variable  was  bui]t  on  a linear 
effects  model  rather  than  the  interacting,  moderator  model.  Ihe  generating 
model  was  y = x +z  +e,  thus  specifying  that  y resulted  from  the  additive 
combination  of  two  unrelated  variables  plus  error.  Again,  20  samples  of  100 
subjects  were  constructed. 

Case  Five.  Ihe  fifth  case  represents  a situation  with  a dependent  variable 
that  is  formed  from  a direct  effect  of  the  independent  variable  plus  a moderator 
effect  plus  error.  'hie  constructive  formula  was  y = x + xz  + e.  There  were 
20  samples  of  100  subjects. 

Case  Six.  In  the  final  case  the  dependent  variable  is  simply  the  independent 
variable  plus  error,  and  the  formula  for  generating  the  dependent  variable  was 
y = x +e.  There  were  20  samples  of  100  subjects  each. 

Table  1 summarizes  the  data  generated  fur  the  six  eases.  Two  hundred 


Insert  Table  1 About  Here 

samples  were  generated.  The  dependent  variable,  y,  was  constructed  differently 
in  the  samples  representing  the  six  different  eases.  Each  case  represents  a 
different  model  of  underlying  effects  In  cases  one,  two,  and  three  the 
dependent  variable  results  from  a moderator  effect  and  random  error.  The 
dependent  variable  is  an  additive  combination  of  the  linear  effects  cf  the 
independent,  moderator,  and  error  variables  in  case  four  where  there  is  no 
moderator  effect.  In  case  five  a combination  of  a direct  linear  effect  of  the 
independent  variable  and  a moderator  effect  (plus  error)  determine  the  dependent 
variable.  Finally,  case  six  presents  a dependent  variable  that  is  a sum  of 
the  independent  variable  and  error  with  no  moderator  effect. 
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Data  Analysis 

* i 

In  each  sample  the  traditional  moderated  regression  analysis  was  conducted 

to  see  if  that  procedure  would  "discover"  moderator  effects  where  they  had  been 

included  in  the  model  and  would  fail  to  "discover"  moderator  effects  where  they 
4 • 2 

were  not  present.  The  increment  in  R was  determined  in  each  sample  as  it 
would  be  in  the  traditional  moderated  regression  analysis  to  represent  the 
increase  in  predictability  controlled  by  the  moderator  effect.  The  statistical 
significance  of  that  increment  was  then  tested  using 


F 


(xz |x,z) 


DO  / \ 

(error  )s 

where  the  numerator  is  the  error  degrees  of  freedom  times  the  sum  of  squares  of 
the  moderator  effect  given  the  linear  effects  of  the  independent  variable  and 
the  moderator  variable,  and  the  denominator  is  the  sum  of  squares  for  error. ^ 


Results 


Statistical  Significance 


As  shown  in  Table  2,  the  moderator  effect  was  "discovered"  by  a requirement 
of  statistical  significance  in  Case  1 and  Case  3.  In  both  of  those  models  all 


Insert  Thble  2 About  Here 


of  the  samples  provided  an  increment  in  R for  the  moderator  effect  that  was 

beyond  that  expected  at  the  .05  level  of  significance.  Though  Case  2 and  Case  5 

also  included  a moderator  effect  in  the  formula  that  generated  the  dependent 

2 

variable,  the  increment  in  R did  not  reach  statistically  significant  size  in 
any  of  the  samples  of  either  of  those  cases.  Case  2 differed  from  Case  1 
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in  that  the  error  component  was  greater  in  Case  2,  i.e. , there  was  more 

unreliability  in  the  dependent  variable.  Case  5 differed  from  Case  1 by  the 

inclusion  of  a direct  relationship  between  the  independent  and  dependent 

2 

variables  as  well  as  the  indirect,  or  moderated,  relationship.  The  R increment 
due  to  the  moderator  was  not  significant  in  any  of  the  samples  of  the  cases 
where  the  generating  model  did  not  include  moderator  effects,  i.e.,  Case  4 and 
Case  6. 

Size  of  'the  Increment 

2 

In  all  of  the  200  samples  the  maximum  increment  in  R attributable  to  the 
moderator  effect  was  .032  in  one  of  the  samples  of  Case  6.  And  there  was  not  a 
moderator  effect  in  the  generating  model  for  the  dependent  variable  of  Case  6! 

The  minimum  increment  of  zero  was  attained  in  three  of  the  cases  including 
Case  5 which  involved  a 'moderator  effect.  In  general , none  of  the  samples, 

2 

either  with  or  without  actual  moderator  effects,  provided  large  increments  in  R . 
Success  of  Moderators  in  Nan-Moderator  Models 

In  the  two  cases  where  there  is  no  moderator  effect  in  the  actual  determina- 
tion of  the  dependent  variable.  Case  4 and  Case  6,  a comparison  was  made 
between  prediction  by  the  moderator  effect  alone  versus  the  true  model.  For 
Case  4,  prediction  of  y by  x plus  z (the  true  model)  provides  a range  of  correla- 
tions over  the  20  samples  from  .716  to  .875  with  a median  correlation  of  .806. 

If  the  prediction  of  y is  by  the  moderator  term  xz,  alone,  the  correlations 
range  from  .708  to  .869  with  a median  of  .801.  In  other  words,  the  moderator 
model  predicts  the  additive  data  nearly  as  well  as  does  the  additive  model.  In 
two  of  the  samples  the  multiplicative  correlation  actually  exceeded  the  additive 
correlation . 
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For  Case  6 the  true  model  does  better  than  the  moderator  term  at  predicting 
the  dependent  variable.  When  y Is  predicted  from  x alone  (the  true  model)  the 
correlations  range  from  .640  to  .803  with  a median  of  .715.  Ihcre  is  no  overlap 
with  the  correlations  using  the  interaction,  xz,  as  the  predictor.  'ihe  moderator 
prediction  gives  a range  of  correlations  from  .359  to  .619  with  a median  of 
.505. 

Discussion 

Using  the  traditional  moderated  regression  analysis  procedures  the 

influence  of  a moderator  effect  was  not  shown  even  when  the  dependent  variable 

was  constructed  to  include  a strong  moderator  effect.  Cases  1,  2,  and  3 created 

the  dependent  variable  as  a moderator  effect  plus  some  random  error.  Under 

those  conditions  Case  2 with  a large  error  component  did  not  demonstrate 

statistical  significance  for  the  moderator  effect,  and  all  three  of  the  cases 

2 

produced  such  small  increments  in  R attributable  to  the  moderator  that  the 
moderator  effect  could  have  been  called  negligible  if  the  size  of  the  increment 
were  interpreted  as  an  indication  of  the  strength  of  the  effect.  In  Case  5, 
where  the  moderator  influence  is  accompanied  by  both  error  and  by  a linear 
effect  of  the  independent  variable,  there  was  neither  statistical  significance 
nor  a sizable  increment.  That  is,  even  in  situations  strongly  biased  in 
favor  of  discovering  a moderator  effect,  i.e.,  a moderator  effect  has  been 
used  in  the  creation  of  the  dependent  variable,  the  tradiLiona.1  moderated 
regression  analysis  leads  to  the  conclusion  of  "no  moderator." 

Why  should  such  a result  occur?  Why  should  this  analysis  be  so  prone  to 
overlook  an  effect  that  is  in  fact  in  the  data?  There  are  two  influences  that 
increase  the  likelihood  of  this  type  II  error. 
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1.  The  hierarchical  nature  of  the  multiple  regression  procedures 

robs  the  moderator  term  of  covari mice  with  the  dependent  variable . 

Ihe  traditional  moderated  regression  analysis  procedure  first  regresses  the 

dependent  variable  on  the  linear  effects  of  the  independent  variable  and  the 

potential  moderator  variable.  The  interaction  term  is  then  regressed  on  only 

2 

the  residual  from  that  regression.  The  increment  in  R which  is  used  to  judge 
both  the  existence  and  the  strength  of  the  moderator  effect  represents  only  the 
relationship  with  that  part  of  the  dependent  variable  variance  that  is  unrelated 
~to-  the... two  linear  effects. 

Any  covariance  with  the  3'epert den-t -va U.ab le  that  is  shared  with  the  linear 
effects  is  automatically  assigned  to  the  linear  effects.  This  implies  a 
preference  for  the  linear  effects  that  is  implicitly  a theoretical  hypothesis 
of  the  generating  effects  that  have  produced  the  dependent  variable.  This 
implied  priority  of  the  main  effects  should  be  supported  by  the  researcher's 
theoretical  understanding  of  the  relationships  among  the  variables;  it  cannot 
be  accepted  simply  because  the  hierarchical  analysis  has  traditionally  proceeded 
in  that  direction.  The  ordering  of  effects  in  the  hierarchical  regression 
analysis  is  arbitrary.  With  appropriate  theoretical  justification  the  data 
analyst  can,  as  validly,  first  assess  the  variance  attributable  to  the  moderator 
effect,  and  then  determine  whether  the  main  effects  make  a significant  and 
sizable  addition  to  the  variance  accounted  for.  The  point  is,  that  the  decision 
of  the  order  in  which  effects  are  assessed  is  one  that  the  researcher  must 


support  theoretically.  Because  the  order  of  the  removal  of  variance  due  to 

effects  is  an  expression  of  a theoretical  preference,  the  investigator  cannot 

. 

abnegate  the  responsibility  for  this  decision  on  the  grounds  of  tradition.  This 
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decision  requires,  not  just  blind  application  of  techniques,  but  scientific 
judgment  and  justification. 

2 . The  linear  regression  model  is  a robust  method  for  accounting 
for  variance. 

Even  if  the  prediction  equation  is  an  imperfect  model  of  the  process  that  generated 
the  data,  the  linear  regression  model  can  often  capture  a great  deal  of  the 
variance.  The  problem  of  using  correlational  procedures  to  assess  the  appro- 
priateness of  an  hypothesized  model  has  been  discussed  in  the  literature  (Alf 
& Abrahams,  1974;  Birnbaum,  1973,  1974;  Rorer,  1974).  Watson  (1972)  has 
demonstrated  that  this  same  problem  can  occur  in  the  analysis  of  variance. 

He  showed  that  data  generated  from  an  interaction  might  be  accounted  for  by 
significant  main  effects  leaving  insignificant  variance  to  be  accounted  for 
by  the  interaction. 

In  a demonstration  similar  to  the  one  presented  in  this  article  Rorer 
(Note  1)  used  a linear  regression  model  to  replicate  the  decisions  that  hja.d  been 
made  with  several  different  decision  strategies  from  constructed  (synthesized) 
data.  Using  11  different  decision  strategies  that  included  linear,  curvilinear, 
configural,  sequential  and  categorical  processes  he  was  able  to  closely 
approximate  the  decisions  of  the  real  functions  with  a simple  linear  function. 
Discussing  a hierarchical  process  for  drawing  conclusions  that  is  the  same,  as 
the  hierarchical  strategy  of  the  traditional  moderated  regression  analysis 
he  states,  "These  results  indicate  vividly  that  it  is  difficult  to  infer 
nonlinearity,  even  when  it  exists"  (p.  12).  "These  results  show  why  the 
search  for  moderator  variables  lias  been  so  fruitless,  just  as  much  as  they 
show  why  human  judges  have  been  unable  to  improve  upon  actuarial  prediction 
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by  means  of  their  purported  ability  to  recognize  and  use  patterned  or  unusual 
relationships.  Even  if  such  relationships  exist,  their  incorporation  into 
the  decision  strategy,  be  it  actuarial  or  subjective,  is  not  going  to  result 
in  much  improvement  over  the  linear  regression  function  in  either  case"  (p.  13). 
llie  linear  regression  model  is  robust  enough  to  capture  much  of  the  predictable 
variance  of  a dependent  variable  even  if  that  dependent  variable  sprang  from 
other  than  linear  origins. 

Hie  critical  distinction  for  the  problem  addressed  in  this  paper  is  the 
difference  between  science  and  application.  If  a research,  has  only  the  applied 
purpose  of  determining  or  demonstrating  how  variance  can  be  accounted  for,  a 
statistical  data  analysis  can  provide  answers.  However,  if  the  research  has 
the  scientific  purpose  of  discovering  or  confirming  an  underlying  model  or 
generating  process  that  produced  the  data,  then  traditional  moderated  regression 
analysis  is  inadequate  for  the  task.  3he  scientific  question  may  be  answered 
in  part  by  a data  analysis  if  the  results  demonstrate  that  a model  is  incompatible 
with  the  data,  but  the  data  analysis  cannot  choose  between  alternative  models 
that  all  fit  the  data.  For  example,  if  y is  predicted  equally  well  by  x +z 
and  by  xz,  the  data  analysis  is  neutral  as  to  choice.  'lhe  selection  of  one 
model  over  the  other  rests  with  the  rational  scientific  judgment  and  preference 
of  the  researcher. 

Ihcre  are  viable  theoretical  contents  that  may  lead  a researcher  to 
support  a moderator  effect  as  the  correct  model  for  his/her  data.  In  his 
presentation  of  the  moderated  regression  analysis  Saunders  (1936)  proposed 
examples  where  a moderator  effect  was  conceptually  supportable.  He  included 
such  examples  as  using  insight  as  a moderator  of  the  relationship  between  self 
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reports  and  more  objective  measures  of  personality  traits  and  using  emotional 
stability  as  a moderator  of  the  relationship  between  academic  ability  measures 
and  academic  success.  Another  interactive  model  that  lias  been  presented  with 
elaborate  theoretical  justifications  is  expectancy  theory  (Staw,  1976;  Vroom, 
1964).  Researchers  in  that  .area  are  likely  to  choose  the  multiplicative 
expectancy  model  for  explanatory  use  even  when  an  additive  model  is  equally 
successful  at  accounting  for  variance.  Their  preference  would  be  based  on 
the  rational  depiction  of  the  relationship  among  the  variables  that  is  afforded 
by  the  expectancy  theory.  Blood  (1977)  presents  a model  of  the  operation 
of  the  cognitive  process  of  self  rewarding  on  work  performance.  The  model 
includes  five  variables  acting  as  moderators.  Hie  logic  of  interactive 
relationships  is  preferred  for  the  model,  because  of  the  nature  of  the  variables 
involved.  This  theoretical  preference  is  rationally  justified. 

The  issue  of  theoretical  parsimony  is  not  involved  in  the  choice  between 
a linear  model  and  a moderator  model.  Whether  two  main  effects  or  one  inter- 
action effect  is  the  more  parsimonious  explanation  is  a matter  of  taste.  Some 
people  will  prefer  an  additive  model  and  other  will  elect  the  multiplicative, 
but  it  is  not  clear  that  one  is  simpler  than  the  other.  In  addition  to  which, 
the  principle  of  parsimony  is  used  to  choose  between  theories  when  other 
considerations  are  equal.  If  other  considerations  lead  to  its  preference,  one 
may  be  justified  in  choosing  a less  parsimonious  model  over  a more  parsimonious 
one . 

This  leaves  us  with  the  important  task  of  suggesting  how  a researcher 
should  proceed  to  assess  the  strength  of  a moderator  effect.  It  there  are 
theoretical  reasons  to  support  a hypothesis  of  a moderator  effect,  the  dependent 
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variable  should  he  directly  repressed  on  the  cross-product  of  the  Independent 
variable  and  the  potential  moderator, 

y = xz.  [A 

Then  if  desired,  the  additional  contribution  of  the  main  effects  can  be  assessed 
by  determining  the.  size  and  significance  of  their  increment  to  the  squared 
multiple  correlation.  That  is,  the  hierarchical  regression  procedure  should 
proceed  in  exactly  the  reverse  order  from  the  traditional  moderated  regression 
analysis  procedure  in  those  cases  where  a moderator  effect  is  theoretically 
justified  (see  Cohen  and  Cohen,  1975  for  analysis  procedures). 

There  may  be  some  cases  where  theory  will  suggest  that  the  independent 
variable  will  influence  the  dependent  variable  partly  through  a main  effect  and 
partly  through  a moderated  effect  (as  in  Case  5).  In  that  event,  a researcher 
may  wish  to  assess  the  size  and  significance  of  the  combined  main  and  moderator 
effects  by  the  regression  of 

y = x + xz.  [5 

Should  this  analysis  be  used,  caution  should  be  exercised  in  interpreting  the 
multiple  correlation  weights.  Because  of  colinearity  influences  these  will  not 
indicate  the  relative  contributions  of  the  main  and  moderator  effects  (see 
Darlington,  1968;  Cohen  and  Cohen,  1975). 

A potential  problem  with  these  suggested  procedures  is  that  they  have 
the  possibility  of  also  encouraging  Type  II  error.  If  a model  is  underfit, 
true  relationships  can  go  undetected  due  to  overestimation  of  the  error  term 
(Johnston,  1972).  Variance  that  is  uniquely  related  to  the  excluded  variables 
is  added  to  the  error  variance.  This  means,  for  example,  that  if  a model  of 
the  form  of  [4  or  [5  were  used  when  the  true  (but  unknown)  model  is  [3,  then 
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the  real  moderator  effect  could  remain  statistically  insignificant  because  of 
this  bias.  This  problem  exists  whenever  a hierarchical  regression  procedure 
is  used  and  whenever  the  mode]  in  use  may  have  excluded  a variable  that 
controls  unique  variance.  This  potential  bias  did  not  cause  a problem  in  the 
present  data,  and  it  appears  to  be.  a less  severe  disturbance  than  the  traditional 
moderated  regression  analysis  procedure. 

Whatever  the  theoretical  predispositions  of  the  investigator,  an  analysis 
should  be  chosen  such  that  it  is  possible  to  confirm  it,  i.e.,  the  analysis 
should  not  virtually  guarantee  type  II  error.  The  theory  should  guide  the 
analysis,  not  the  reverse.  In  the  hierarchical  analysis  traditionally  used  to 
discern  moderated  regression,  it  is  not  possible  to  rule  out  a moderator  as  a 
generating  factor  for  the  dependent  variable  by  showing  that  the  main  effects  alone 
can  account  for  most  of  the  variance.  Thoughtlessly  following  the  traditional 
analytic  procedures  cannot  replace  the  exercise  of  scientific  judgment  based  on 
carefully  drawn  theory. 
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Nhis  increment  in  called  the  squared  scnli partial  correlation  coefficient 
by  Cohen  and  Cohen  ((975)  and  the  "Usefulness"  index  by  Darlington  (1968). 

9 

Because  of  interdependencies  among  Lite  variables,  these  wore  not  12 
independent  effects. 

It  is  important  to  note  that  the  distr.ibuLi.ons  of  independent  and 
moderator  variables  included  only  positive  numbers.  This  is  usual  for  social 
science  data.  Because  of  the  manner  of  construction  of  y,  the  results  would 
be  changed  if  x and  z took  on  both  positive  and  negative  values. 

N'he  authors  wish  to  thank  Mark  Elliott  and  Julie  Merer  for  l heir  help 
with  the  data  generation  and  analysis  for  this  study. 

This  is  the  sum  of  squares  for  error  of  this  prediction  model,  not  sum 
of  squares  for  the  "e"  term  used  in  generating  the  dependent  variable. 


Tabic?  1 


Structure  of  the  Simulated  Samples 
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Table  2 

Results  of  Traditional  Moderated  Regression  Analysis 


Average 
Increment 
in  R?- 

Minimum 
I ne remen t 
in  R2 

Maximum 
1 nc remen t 
in  1‘2 

Increments 
Si j;n  j f i cant 
at  .05  level 

.005 

.002 

.009 

100  of  100 

.006 

.001 

.022 

0 of  20 

.006 

.003 

.010 

20  of  20 

vOOA 

.000 

.015 

0 of  20 

.003 

.000 

.016 

0 of  20 

.008 
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